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Foreword 

Flying  insects  display  remarkable  agility,  despite  their  diminutive  eyes  and  brains.  Over  the  past 
two  decades  the  has  been  increasing  interest  in  investigating  how  insects  use  their  vision  to 
stabilize  flight,  regulate  flight  speed  and  altitude,  avoid  collisions  with  objects,  and  detect  and 
intercept  other  moving  targets  (Srinivasan,  2011)  [1].  This  project,  funded  by  the  US  Army 
Research  Office,  is  part  of  a  larger  and  continuing  effort  in  our  laboratory  to  understand  the 
mechanisms  of  visual  guidance  in  flying  insects,  and  to  use  this  biological  inspiration  to  design 
novel  algorithms  for  the  guidance  of  autonomous  aerial  vehicles. 
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Figure  7  (page  11)  Comparison  of  pitch  (upper  panel)  and  roll  (centre  panel)  as  computed  by  the  vision 
system,  with  the  corresponding  values  computed  from  the  gyroscopic  signals.  The  bottom  panel  shows 
the  altitude,  as  computed  by  the  vision  system. 

Figure  8  (page  12)  Performance  of  system  in  controlling  altitude  and  attitude  (a)  Altitude  (black  trace) 
and  pitch  (blue  trace)  during  periods  of  manual  flight  (white  background)  and  automatic  flight  (pink 
background).  During  the  auto  mode  the  system  is  commanded  to  hold  the  aircraft  at  an  altitude  of  10  m 
and  a  pitch  of  0  deg.  (b)  View  of  aircraft  (left  panel)  and  image  acquired  by  the  front  camera  (right  panel) 
during  an  auto  mode  in  which  the  aircraft  is  commanded  to  maintain  an  altitude  of  15  m  and  a  roll  angle 
of  -45  deg. 

Figure  9  (page  13)  Illustration  of  how  the  terrain-following  vision  system  of  Fig.  1  defines  a  collision- 
free  cylinder  whose  radius  depends  upon  the  highest  magnitude  of  optic  flow  that  is  measured. 

Figure  10  (page  15)  Insect  ocellus-inspired  system  for  measuring  and  stabilizing  aircraft  attitude  by 
monitoring  the  horizon,  (a)  Illustration  of  system  performance.  The  graphs  show  pitch  (green  trace)  and 
roll  (red  trace)  during  the  manual  flight  mode  (blue  trace  low),  and  during  the  auto  mode  (blue  trace  high) 
when  the  system  is  commanded  to  hold  a  pitch  angle  of  0  deg  and  a  roll  angle  of  40  deg.  (b)  The  left-hand 
panel  shows  an  image  captured  by  the  vision  system  when  the  aircraft  is  under  automatic  control  and 
commanded  to  maintain  the  pitch  and  roll  angles  indicated  above.  The  right-hand  panel  shows  an 
animation  of  the  orientation  of  the  aircraft  relative  to  the  ground  plane  at  the  same  instant  of  time. 

Figure  11  (page  17)  (a)  Robotic  gantry  for  testing  moving  object  detection  algorithms  (b)  Panoramic 
imaging  system  comprising  a  camera  (left)  and  a  specially  profiled  reflective  surface  (right)  (c)  Raw 
image  recorded  by  the  system  (d)  Unwrapped  image 
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Figure  12  (page  17)  Experimental  configuration  for  testing  the  detection  of  a  small  target  moving  from 
the  bottom  of  the  image  to  the  top  (1)  or  from  the  top  to  the  bottom  (2)  while  the  vision  system  itself 
moves  continuously  toward  the  top  (i.e.  in  the  same  direction  as  (1)) 

Figure  13  (page  18)  Test  image  sequences  for  detecting  a  small  target  moving  from  the  bottom  of  the 
image  to  the  top  (upper  row)  and  from  the  top  to  the  bottom  (lower  row)  while  the  vision  system  itself 
moves  toward  the  top  at  a  constant  speed. 

Figure  14  (page  18)  Tests  of  moving  target  detection  by  a  moving  vision  system.  The  graph  shows  the 
fraction  of  frames  in  which  the  moving  target  is  detected  for  various  object  speeds,  using  (1)  a  DOG  filter 
to  extract  optic  flow  contrast  and  (2)  a  DOG  filter  in  conjunction  with  a  model  that  assumes  a  planar 
background.  The  graph  also  compares  the  results  obtained  with  and  without  the  use  of  a  scheme  to  predict 
the  target  location  in  the  next  frame  on  the  basis  of  the  measured  object  velocity  between  the  previous  two 
frames. 

Figure  15  (page  19)  Position  errors  in  the  detection  of  the  location  of  a  moving  target  by  a  moving  vision 
system,  at  various  object  speeds.  The  graph  shows  position  error  (in  pixels)  using  (1)  a  DOG  filter  to 
extract  optic  flow  contrast  and  (2)  a  DOG  filter  in  conjunction  with  a  model  that  assumes  a  planar 
background. 

Figure  16  (page  19)  False  positives  in  the  detection  of  the  location  of  a  moving  object  by  a  moving 
vision  system,  at  various  object  speeds.  The  graph  shows  position  error  (in  pixels)  using  (1)  a  DOG  filter 
to  extract  optic  flow  contrast  and  (2)  a  DOG  filter  in  conjunction  with  a  model  that  assumes  a  planar 
background. 

Figure  17  (page  20)  Input  scene,  with  the  observer  moving  along  a  linear  trajectory  at  a  constant  speed. 
The  scene  contains  a  static  object  (right)  and  an  object  undergoing  self-motion  (left). 

Figure  18  (page  20)  Optic  flow:  the  magnitudes  of  the  computed  optic  flow  vectors. 

Figure  19  (page  21)  Stereo  disparity:  the  computed  stereo  disparity  for  the  scene.  Note  that  the  objects 
are  at  about  the  same  distance  as  the  background  plane. 

Figure  20  (page  21)  Moving  object  detection:  The  self-moving  object  is  automatically  detected  by 
combining  information  from  the  measured  optic  flow  and  stereo  disparity  fields. 
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Statement  of  the  problem  studied 

The  aim  of  this  project  is  to  design,  develop  and  field-test  biologically  inspired  strategies, 
algorithms  and  hardware  for  vision-based  guidance  of  aircraft  to  achieve  a  variety  of  tasks,  listed 
below. 

i)  Control  of  aircraft  altitude  and  attitude,  terrain  following,  and  obstacle  avoidance 

ii)  Horizon-based  stabilization  and  control  of  aircraft  attitude 

iii)  Detection  of  moving  targets  by  a  moving  vision  system,  using  algorithms  based  on 
analysis  of  optic  flow. 
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Summary  of  the  most  important  results 

i)  Control  of  aircraft  altitude  and  attitude  by  using  ‘virtual  optic  flow’ 

Flying  at  a  constant,  low  height  above  the  ground  is  important  when  there  is  a  need  to  perform 
close-up  photographic  exploration  of  terrain,  or,  in  a  military  application,  to  evade  detection  by 
enemy  radar.  If  the  ground  speed  of  the  aircraft  is  known  (e.g.  through  measurement  of 
airspeed,  or  from  GPS  information),  then,  following  the  example  of  the  honeybee  [1,  2],  the 
height  above  the  ground  can  be  computed  and  regulated  by  measuring  the  optic  flow  that  is 
generated  by  the  image  of  the  ground.  The  optic-flow  based  approach  is  attractive  because  it  only 
requires  the  presence  of  a  small,  inexpensive,  low-resolution  video  camera  on  board.  This  is  in 
contrast  to  traditional  methods  of  height  measurement  that  use  heavy,  bulky  and  power  hungry 
instrumentation  such  as  radar  or  ultrasound,  radiate  energy,  and  compromise  stealth. 

Optics  of  the  vision  system 

During  high  speed  flight  at  low  altitudes,  the  image  of  the  ground,  as  imaged  by  a  downward¬ 
looking  camera,  can  move  at  a  very  high  velocity,  complicating  the  measurement  of  optic  flow. 
To  address  this  problem,  we  have  designed  and  developed  a  vision  system  that  uses  a  camera  and 
a  specially  shaped  convex  mirror  profile  as  shown  in  Figs.  la,b  to  achieve  two  objectives.  They 
are:  (i)  To  reduce  the  speed  of  the  image  of  the  ground  to  values  that  are  low  enough  to  permit 
accurate  measurement;  and  (ii)  To  remove  the  perspective-induced  distortion  of  different  regions 
of  the  ground  in  front  of  the  aircraft  (see  Fig.  lc).  This  ensures  that  there  is  no  variation  in  image 
velocity  along  the  vertical  axis  of  the  remapped  image  (Fig.  Id),  thus  promoting  the  accurate 
measurement  of  optic  flow  along  this  axis  [3]. 

This  way  of  mapping  the  world  is  similar  in  some  respects  to  that  achieved  by  the  compound 
eyes  of  semi-terrestrial  crabs  that  live  on  a  flat  substrate  [4]—  although  the  objective  in  the  case 
of  the  crab  seems  to  be  translate  range  measurements  into  angular  measurements  in  the  eye, 
rather  than  to  measure  image  velocities. 
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Figure  1  Illustration  of  vision  system  for  terrain  following,  (a)  A  video  camera  views  the  ground  through 
a  specially  shaped,  curved  reflecting  surface  (b)  View  of  system  installed  on  the  underside  of  a  model 
aircraft  (c)  Simulation  of  view  in  the  mirror  as  captured  by  the  camera  during  horizontal  flight  over  an 
infinite  horizontal  plane  (d)  Unwarped  version  of  (c),  remapped  as  in  Fig.  3b,  showing  the  removal  of  the 
perspective  distortion  along  the  vertical  axis. 


When  using  optic  flow  to  compute  the  range  of  an  object,  it  is  important  to  bear  in  mind  that  it  is 
only  the  translation-induced,  component  of  optic  flow  that  contains  information  on  range  - 
nearer  objects  induce  higher  magnitudes  of  optic  flow.  The  flows  induced  by  any  accompanying 
rotations  of  the  aircraft  -  for  example,  yaw,  pitch  or  roll  -  do  not  carry  information  on  range, 
because  their  magnitudes  are  independent  of  range.  Consequently,  we  cannot  work  directly  with 
the  raw  optic  flow  readings.  It  is  necessary  to  (i)  measure  the  rates  of  yaw,  roll  and  pitch  using 
gyroscopes  (ii)  use  the  gyroscopic  readings  in  conjunction  with  pre-computed  optic-flow 
templates  for  yaw,  roll  and  pitch  to  determine  the  rotation-induced  flow  components  and  (iii) 
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calculate  the  translation-induced  component  of  optic  flow  by  subtracting,  or  ‘peeling  off’  the 
flow  components  that  are  generated  by  the  rotations.  The  residual  pattern  of  optic  flow,  which  is 
purely  that  induced  by  the  translational  component  of  the  aircraft’s  motion,  can  then  be  used 
compute  range. 

We  have  developed  a  simpler  solution  to  the  above  problem.  This  is  to  artificially  create  the 
effect  of  a  purely  translatory  forward  motion  of  the  aircraft  by  configuring  two  of  the 
camera/mirror  systems  of  Fig.  1,  as  shown  in  Fig.  2.  The  apparent  optic  flow  that  is  measured  by 
comparing  the  images  acquired  by  these  two  systems  at  the  same  instant  of  time  is  then  exactly 
equal  to  the  flow  that  would  have  been  registered  by  the  rear  mirror  system  if  it  (and  the  aircraft) 
had  made  a  pure  translation  to  the  position  of  the  front  mirror  system.  The  ‘virtual  optic  flow’ 
generated  by  this  coaxially  arranged,  dual  camera-mirror  system  provides  an  accurate  and 
immediate  reproduction  of  the  translation-induced  component  of  the  flow,  without  having  to 
move  the  aircraft  at  all.  This  flow  can  then  be  used  for  a  variety  of  purposes,  including  (i) 
computing  a  range  map  of  the  environment,  (ii)  fitting  a  plane  to  the  range  data  to  estimate  the 
distance  and  the  orientation  of  the  ground  plane  relative  to  the  aircraft  (iii)  using  this  information 
in  a  feedback  loop  to  control  the  aircraft’s  altitude  and  attitude. 


Camera  1  Mirror  1  Camera  2  Mirror  2 


Figure  2  Illustration  of  twin  coaxial  camera/mirror  system  for  measuring  virtual  translational  optic  flow. 
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Calibration  of  the  vision  system 

Fig.  3a  shows  a  view  of  the  laboratory  test  arena  in  which  the  device  was  calibrated,  and  Fig.  3b 
shows  the  virtual  optic  flow  computed  between  the  images  captured  by  cameras  1  and  2.  Nearer 
points  of  the  environment  generate  greater  magnitudes  of  virtual  optic  flow. 

Fig.  4a  shows  the  range  readings  computed  by  the  vision  system  for  a  vertical  cross  section  of 
the  arena,  and  Fig.  4b  shows  a  reconstruction  of  the  terrain  based  on  the  range  readings.  The 
error  in  the  range  readings  is  less  than  5%  [5]  [6]. 


Figure  3  (a)  Test  arena  for  calibration  of  the  vision  system  (b)  Virtual  translational  optic  flow  measured 
from  the  images  captured  by  the  two  coaxial  cameras  of  the  vision  system 


Figure  4  (a)  Range  as  a  function  of  (roll)  view  angle  (blue  curve  and  SD  bars)  as  computed  by  the  vision 
system  at  one  cross  section  of  the  arena,  compared  with  the  true  range  (grey  curve)  (b)  reconstructed 
terrain  profile  of  the  area,  compared  with  the  actual  profile,  which  is  represented  by  the  boundary  planes 
of  the  co-ordinate  axes. 
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This  system  is  used  to  estimate  the  aircraft’s  height  above  ground.  If  we  assume  the  ground  to 
be  approximately  planar  and  horizontal,  we  can  also  use  the  range  readings  to  estimate  the 
orientation  of  the  aircraft  relative  to  the  ground,  and  therefore  its  attitude  in  relation  to  pitch  and 
roll.  Fig.  5  illustrates  the  magnitude  profile  of  the  virtual  optic  flow  (or,  effectively  disparity) 
that  would  be  generated  in  the  vision  system  by  the  ground  during  horizontal  flight  at  zero  pitch 
and  roll.  The  measured  virtual  optic  flow  magnitudes  are  fitted  to  a  planar  model  of  the  ground  to 
estimate  the  roll  and  pitch  of  the  aircraft  [5]. 


Figure  5  Magnitude  of  the  virtual  optic  flow  that  would  be  generated  in  the  vision  system  by  the  ground 
during  horizontal  flight  at  zero  pitch  and  roll. 

Field  tests  of  vision  system 

Fig.  6  shows  a  view  of  the  vision  system,  mounted  on  the  nose  cone  of  a  model  aircraft  -  a  Super 
Frontier  Senior-46  (wingspan  2040mm). 


Figure  6  View  of  vision  system  mounted  on  the  nose  cone  of  a  model  aircraft 
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Fig.  7  shows  the  results  of  a  flight  test  of  the  system.  The  graphs  compare  the  roll  and  pitch,  as 
computed  by  the  vision  system,  with  the  roll  and  pitch  as  reported  by  the  gyros.  The  agreement  is 
good,  allowing  for  the  fact  that  gyro  readings  are  not  100%  reliable  because  of  drift  [5].  Hence, 
in  estimating  attitude  the  vision  system  that  we  have  developed  is  likely  to  be  more  accurate  than 
the  gyros. 


Figure  7  Comparison  of  pitch  (upper  panel)  and  roll  (centre  panel)  as  computed  by  the  vision  system, 
with  the  corresponding  values  computed  from  the  gyroscopic  signals.  The  bottom  panel  shows  the 
altitude,  as  computed  by  the  vision  system. 
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Closed-loop  control  of  altitude  and  attitude 


The  next  step  was  to  test  the  performance  of  the  vision  system  in  closed  loop  [7].  Fig.  8a  shows 
the  performance  of  the  system  in  regulating  altitude  and  attitude,  where  control  alternates 
between  the  manual  mode  (when  the  aircraft  is  controlled  by  a  pilot)  and  the  auto  mode  (when 
the  autopilot  is  in  operation).  In  the  auto  mode  the  system  is  commanded  to  hold  the  altitude  at 
10m  and  the  pitch  at  0  deg.  It  is  evident  from  Fig.  8a  that  the  system  performs  these  functions 
well.  When  control  is  passed  from  manual  to  auto  at  a  height  of  20  m  and  a  pitch  of  -35  deg,  the 
system  is  able  to  attain  the  target  parameters  within  about  3  seconds.  The  left-hand  panel  of  Fig. 
8b  is  a  view  of  the  aircraft  flying  in  a  different  auto  mode  where  it  is  commanded  to  maintain  an 
altitude  of  15m  and  a  roll  angle  of  -45  deg.  The  right-hand  panel  shows  an  image  captured  by  the 
front  camera  at  the  same  instant  of  time. 


Figure  8  Performance  of  system  in  controlling  altitude  and  attitude  (a)  Altitude  (black  trace)  and  pitch 
(blue  trace)  during  periods  of  manual  flight  (white  background)  and  automatic  flight  (pink  background). 
During  the  auto  mode  the  system  is  commanded  to  hold  the  aircraft  at  an  altitude  of  10  m  and  a  pitch  of  0 
deg.  (b)  View  of  aircraft  (left  panel)  and  image  acquired  by  the  front  camera  (right  panel)  during  an  auto 
mode  in  which  the  aircraft  is  commanded  to  maintain  an  altitude  of  15  m  and  a  roll  angle  of  -45  deg. 
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Obstacle  detection  and  avoidance 

The  mapping  performed  by  the  mirror  shown  in  Fig.  1  has  an  additional  advantage:  the  largest 
magnitude  of  the  virtual  optic  flow  that  is  sensed  by  the  system  defines  the  radius  of  a  ‘collision- 
free’  cylinder  of  space  (a  clear  zone)  through  which  the  aircraft  can  fly  safely  (Fig.  9). 


Collision- 


Figure  9  Illustration  of  how  the  terrain-following  vision  system  of  Fig.  1  defines  a  collision-free  cylinder 
whose  radius  depends  upon  the  highest  magnitude  of  optic  flow  that  is  measured. 

The  smaller  the  flow  magnitude,  the  larger  the  radius  of  this  cylinder,  and  the  safer  the  passage. 
Collisions  with  obstacles  can  be  avoided  by  defining  a  collision-free  cylinder  of  a  prescribed 
radius,  and  computing  the  magnitude  of  optic  flow  corresponding  to  this  radius.  Any  object  that 
generates  optic  flow  of  a  magnitude  greater  than  this  criterion  value  will  penetrate  the  collision- 
free  cylinder  and  be  flagged  as  an  obstacle.  Control  is  then  applied  to  veer  away  from  the 
obstacle,  thus  moving  it  outside  the  cylinder  [7],  An  example  of  collision  avoidance  using  this 
technique  can  be  viewed  in  the  movie  in  the  accompanying  Powerpoint  presentation. 

ii)  Horizon-based  stabilization  and  control  of  attitude 

Conventional  methods  of  estimating  and  stabilizing  attitude  involve  the  use  of  gyroscopes  to 
measure  the  rates  of  roll  and  pitch.  This  approach  has  the  disadvantage  that  the  instantaneous 
attitude  is  determined  by  integrating  these  rates  over  time,  which  means  that  the  error  in  the 
estimate  of  the  attitude  will  increase  with  time,  due  to  the  presence  of  noise  in  the  rate  signals. 
The  problem  does  not  arise  if  the  horizon  is  used  to  estimate  attitude,  because  this  visual  feature 
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provides  an  absolute  external  reference  and  there  is  no  cumulative  error  arising  from  integrating 
angular  rates  over  time. 

Flying  insects  control  their  attitude  in  roll  and  pitch  by  using  three  visual  sensors,  known  as 
ocelli,  to  monitor  the  horizon  [8]  [9].  We  have  incorporated  this  principle  into  a  vision  system, 
as  detailed  below,  that  is  capable  of  monitoring  roll  and  pitch  accurately  and  robustly,  regardless 
of  the  position  of  the  sun,  clouds  or  other  confounding  factors. 

We  have  developed  an  algorithm  to  locate  the  profile  of  the  horizon  in  the  image  captured  by  one 
or  more  on-board  cameras.  The  colour  of  each  pixel  in  the  image  is  analysed  to  determine 
whether  it  belongs  to  the  ground  or  to  the  sky  [10]  [11].  This  determination  is  carried  out  by 
generating  a  signal-derived  variable,  U,  which  is  an  optimally  weighted  linear  combination  of  the 
R,  G  and  B  colour  components  of  the  pixel.  The  pixel  is  declared  to  belong  to  the  sky  or  the 
ground  according  to  whether  U  is  greater  or  lower  than  a  preset  threshold.  This  procedure 
enhances  the  speed  of  discrimination  and  maximises  its  accuracy  [11].  Horizon  pixels  are 
identified  as  those  lying  at  the  transition  between  ground  and  sky.  The  positions  of  these  horizon 
pixels  in  the  image  are  then  back-projected  into  the  external  environment  using  the  known 
geometry  of  the  camera’s  optics,  and  a  3-D  plane  is  fitted  to  these  points  as  detailed  in  [11].  The 
orientation  of  this  plane  relative  to  the  vision  system  then  specifies  the  attitude  of  the  aircraft 
relative  to  the  horizon,  thus  determining  its  pitch  and  roll.  This  scheme  is  robust  to  the  position 
of  the  sun  and  to  changes  in  illumination  because  it  locates  the  horizon  by  using  information  on 
colour,  rather  than  intensity.  A  further  advantage  of  this  system  is  that  it  does  not  require  a 
separate  set  of  optoelectronic  sensors  for  monitoring  aircraft  attitude  -  the  algorithm  can  be 
incorporated  into  a  vision  system  that  is  already  place,  e.g.  for  sensing  optic  flow. 

The  performance  of  this  scheme  is  illustrated  in  Fig.  10a,  which  shows  the  results  of  a  test  in 
which  the  aircraft  was  initially  flown  manually,  and  subsequently  commanded  to  hold  a  pitch 
angle  of  0  deg  and  a  roll  angle  of  40  deg  during  the  time  interval  38  -  107  sec.  Manual  control 
was  resumed  after  107  sec.  It  is  evident  that  the  aircraft  maintains  the  prescribed  attitude  in  a 
stable  and  accurate  manner  during  the  period  of  automatic  control.  The  left-hand  panel  of  Fig. 
10b  shows  one  frame  captured  by  the  camera  during  the  auto  mode,  with  the  red  line  showing 
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the  estimated  horizon  profile  and  the  numbers  indicating  the  computed  values  of  roll  and  pitch. 
The  right-hand  panel  of  Fig.  10b  shows  an  animated  reconstruction  of  the  instantaneous 
orientation  of  the  aircraft  relative  to  the  ground  plane.  Recent  work  has  demonstrated  that  this 
technique  for  monitoring  and  controlling  aircraft  attitude  can  be  used  to  automate  the  execution 
of  a  variety  of  aerobatic  manoeuvres  such  as  loops  and  Immelmann  turns  [12]. 


b 

Figure  10  Insect  ocellus-inspired  system  for  measuring  and  stabilizing  aircraft  attitude  by  monitoring  the 
horizon,  (a)  Illustration  of  system  performance.  The  graphs  show  pitch  (green  trace)  and  roll  (red  trace) 
during  the  manual  flight  mode  (blue  trace  low),  and  during  the  auto  mode  (blue  trace  high)  when  the 
system  is  commanded  to  hold  a  pitch  angle  of  0  deg  and  a  roll  angle  of  40  deg.  (b)  The  left-hand  panel 
shows  an  image  captured  by  the  vision  system  when  the  aircraft  is  under  automatic  control  and 
commanded  to  maintain  the  pitch  and  roll  angles  indicated  above.  The  right-hand  panel  shows  an 
animation  of  the  orientation  of  the  aircraft  relative  to  the  ground  plane  at  the  same  instant  of  time. 
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iii)  Detection  of  moving  targets 


While  it  is  relatively  straightforward  to  detect  a  moving  object  in  a  scene  when  the  observer  (or 
camera)  is  stationary  -  for  example,  through  frame  subtraction  -  this  task  becomes  much  more 
complex  when  the  observer  is  in  motion.  The  reason  is  that,  when  the  observer  moves,  the 
images  of  the  object  as  well  as  the  background  are  in  motion.  Relative  motion  between  the  object 
and  the  background  is  not  a  reliable  cue  for  detecting  a  moving  object.  This  is  because  the  image 
of  a  stationary  object  can  move  relative  to  the  image  of  its  background  if  the  observer  is 
undergoing  translatory  motion  and  the  object  is  nearer  to  the  observer  than  is  the  background. 

We  have  developed  and  tested  two  classes  of  algorithms  for  detection  of  a  moving  target  by  a 
moving  vision  system. 

1.  One  class  of  algorithms  involves  (a)  measuring  the  optic  flow  field  in  the  entire  scene  and  (b) 
detecting  the  presence  of  the  moving  target  through  the  motion  contrast  that  it  presents  against  its 
immediate  background.  This  motion  contrast  is  detected  by  applying  a  Difference-of-Gaussian 
(DOG)  filter  to  the  optic  flow  field.  The  technique  is  refined  by  (i)  assuming  a  planar  model  for 
the  structure  of  the  background  and  (b)  predicting  the  likely  location  of  the  target  in  each  frame 
from  information  on  its  position  and  velocity  as  measured  during  the  past  two  frames. 

The  results  of  this  approach  are  shown  in  the  next  6  figures  and  described  in  the  legends. 
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Figure  11  (a)  Robotic  gantry  for  testing  moving  object  detection  algorithms  (b)  Panoramic  imaging 
system  comprising  a  camera  (left)  and  a  specially  profiled  reflective  surface  (right)  (c)  Raw  image 
recorded  by  the  system  (d)  Unwrapped  image 
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Figure  12  Experimental  configuration  for  testing  the  detection  of  a  small  target  moving  from  the  bottom 
of  the  image  to  the  top  (1)  or  from  the  top  to  the  bottom  (2)  while  the  vision  system  itself  moves 
continuously  toward  the  top  (i.e.  in  the  same  direction  as  (1)) 
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(a)  Start  frame 


(b)  Middle  frame 


(c)  End  frame 


(a)  Start  frame  (b)  Middle  frame  (c)  End  frame 


Figure  13  Test  image  sequences  for  detecting  a  small  target  moving  from  the  bottom  of  the  image  to  the 
top  (upper  row)  and  from  the  top  to  the  bottom  (lower  row)  while  the  vision  system  itself  moves  toward 
the  top  at  a  constant  speed. 


Figure  14  Tests  of  moving  target  detection  by  a  moving  vision  system.  The  graph  shows  the  fraction  of 
frames  in  which  the  moving  target  is  detected  for  various  object  speeds,  using  (1)  a  DOG  filter  to  extract 
optic  flow  contrast  and  (2)  a  DOG  filter  in  conjunction  with  a  model  that  assumes  a  planar  background. 
The  graph  also  compares  the  results  obtained  with  and  without  the  use  of  a  scheme  to  predict  the  target 
location  in  the  next  frame  on  the  basis  of  the  measured  object  velocity  between  the  previous  two  frames. 
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Figure  15  Position  errors  in  the  detection  of  the  location  of  a  moving  target  by  a  moving  vision  system,  at 
various  object  speeds.  The  graph  shows  position  error  (in  pixels)  using  (1)  a  DOG  filter  to  extract  optic 
flow  contrast  and  (2)  a  DOG  filter  in  conjunction  with  a  model  that  assumes  a  planar  background. 


Figure  16  False  positives  in  the  detection  of  the  location  of  a  moving  object  by  a  moving  vision  system, 
at  various  object  speeds.  The  graph  shows  position  error  (in  pixels)  using  (1)  a  DOG  filter  to  extract  optic 
flow  contrast  and  (2)  a  DOG  filter  in  conjunction  with  a  model  that  assumes  a  planar  background. 
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2.  A  second  class  of  algorithms  that  we  have  investigated  involves  (a)  measuring  the  optic  flow 
field  that  is  generated  in  the  entire  scene  by  a  pure  translation  of  the  vision  system  and  (b) 
computing  a  range  map  for  the  entire  scene  using  stereo  vision.  Moving  targets  are  then  detected 
as  locations  where  the  measured  optic  flow  field  vector  does  not  match  the  vector  that  is 
predicted  from  the  measured  range  map  and  the  known  direction  of  translation. 

The  results  of  this  approach  are  shown  in  the  next  4  figures  and  described  in  the  legends. 


Figure  17  Input  scene,  with  the  observer  moving  along  a  linear  trajectory  at  a  constant  speed.  The  scene 
contains  a  static  object  (right)  and  an  object  undergoing  self-motion  (left). 


Figure  18  Optic  flow:  the  magnitudes  of  the  computed  optic  flow  vectors. 
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Figure  19  Stereo  disparity:  the  computed  stereo  disparity  for  the  scene.  Note  that  the  objects  are  at  about 
the  same  distance  as  the  background  plane. 


Figure  20  Moving  object  detection:  The  self-moving  object  is  automatically  detected  by  combining 
information  from  the  measured  optic  flow  and  stereo  disparity  fields. 
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